Frame rate and viseme analysis for multimedia applications

In the future multimedia technology will be able to provide video frame rates equal to or better than 30 frames-per-second (FPS). Until that time the hearing impaired community will be using band-limited communication systems over un-shielded twisted pair copper wiring. As a result multimedia communication systems will use a coder/decoder (CODEC) to compress the video and audio signals for transmission. For these systems to be usable by the hearing impaired community, the algorithms within the CODEC have to be designed to account for the perceptual boundaries of the hearing impaired. In this paper we investigate the perceptual boundaries of speechreading and multimedia technology, which are the constraints that e ect speechreading performance. We analyze and draw conclusions on the relationship between viseme groupings, accuracy of viseme recognition, and presentation rate. These results are critical in the design of multimedia systems for the hearing impaired.